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What are reporting guidelines 
and how can they help address 
deficiencies in reporting? 


Recap: Incomplete/unclear reporting 


Whatis missing from descriptions of [Exercise prescription: a case for standardised 
treatment in trials and reviews? reporting 


Replicat ng non-pharmacological treatments in practice depends on how well they Clin Chem Lab Med 20125013) 411-413 © 2012 by Walter de Gruyter - Berlin + Boston DO! 10. 1515/cclm 2011 0904 
have been described in research studies, say Paul Glasziou and colleagues 











Have yoe ever read a uial a e el e o =! An appeal to medical journal editors: the need for 
a: Di AR : a full description of laboratory methods and specimen 
Akhough ( ONSORT and related ai- labed a uspplere Lan arch with a mote į qua handling in clinical study reports 


0 PLOS| ov 
Adequacy of Published Oncology Randomized 
RESEARCH ARTICLE Controlled Trials to Provide Therapeutic Details 


The Devil Is in the Details: Incomplete Needed for Clinical Application 
Reporting in Preclinical Animal Research | 


Marc T. Avey"2*, David Moher", Katrina J. Sullivan’, Dean Fergusson’, Gilly Griffin’) Jennifer M. Duff, Helen Leather, Edmund O. Walden, Kourtney D. LaPlant and Thomas J. George Jr 
Jeremy M. Grimshaw", Brian Hutton", Manoj M. Lalu’, Malcolm Macleod’, 
F.Turgeon*", Lauralyn Mcintyre'**!, Canadian Critical Care Translational Biology G 


Reproducibility of Search Strategies Is Poor in Systematic | Non-compliance with randomised allocation and missing outcome 
Reviews Published in High-Impact Pediatrics, Cardiology data in randomised controlled trials evaluating surgical interventions: 
and Surgery Journals: A Cross-Sectional Study a systematic review 





Jonathan B. Koffel DE. Melissa L. Rethlefsen co Temitope E. Adewuyi Mi, Graeme MacLennan and Jonathan A. Cook 


Published: September 26, 2016 + http://dx.doi.org/10.1371/journal.pone.0163309 BMC Research Notes 2015 8:403 | DOI: 10.1186/s13104-015-1364-9 | O Adewuyi et al. 2015 








Recap: Quality of reporting in systematic reviews 


Volume 31 , Issue 2 Applicable or non-applicable: investigations of 
March/April 2016 o. a , : 
Pages 338-351 clinical heterogeneity in systematic reviews 


Quality Assessment of Systematic Reviews on Oral Implants || '*""? E- “ness and Joel]. Gagnier = 


BMC Medical Research Methodology BMC series - open, inclusive and trusted 2016 16:19 
Momen A. Atieh, BDS, MSc, DClinDent, PhD/Warwick J. Duncan, BDS, MDS, || DOI: 10.1136/s12874-016-0121-7 © Chess and Gagnier. 2016 








Compliance of Systematic Reviews in Plastic 


Surgery With the PRISMA Statement | ) cin Epidemiol. 2016 Jan 13. pi: 50895-4355(16)00038-1. doi: 10.10164 jelinepi.2016.01.008. [Epub a 


Seon-Young Lee, BMedSc*; Harkiran Sagoo, BSc(Hons»; Katharine Whitehurst, E Strong heterogeneity of outcome reporting in systematic reviews. 
Georgina Wellstead, BSc(Hons)*; Alexander J. Fowler, BSc(Hons), MBBS*; Riaz A 


(Oxf), MRCSEng, FHEA, FRSPH®; Dennis Orgill, MD, PhD” 








Sautenet B', Contentin L?, Bigot A*, Giraudeau B+. 







Systematic reviews experience major limitations in 
reporting absolute effects Risk of Bias in Systematic Reviews of Non-Randomized 
Studies of Adverse Cardiovascular Effects of 
hiazolidinediones and Cyclooxygenase-2 Inhibitors: 
PA of a New Cochrane Risk of Bias Tool 









Pablo Alonso-Coello=4 z Alonso Carrasco-Labra, Romina Brignardello-Pete 
A. Akl, Robin W.M. Vernooij, Brad C. Johnston, Xin Sun, Matthias Briel, Jason 
Carlos E. Granados, Alfonso lorio, Affan Irfan, Laura Martinez Garcia, Reem A 
Morera, Anna Selva, Ivan Sola, Andrea Juliana Sanabria, Kari A.O. Tikkinen, f 
ger Schünemann, Gordon H. Guyatt 


A third of systematic reviews changed or did nai y the as daa rr rated 
primary outcome: A PROSPERO register study 
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Andrea C. Tricco*- >. @- Má, Elise Cogo*, Matthew J. Page”, Julie Polisena® =, Alison Booth’, Kerry 
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How to improve reporting 


Reporting 


guidelines 
Research 


Guidance on research 


methods: GCP, GEP... > 
Translation Publication 


| 


Guidance on 
scientific writing & 
Author Instructions 


Dissemination 








What are reporting guidelines? 


Statements that provide advice on how to report research methods 
and findings 


Specify a minimum set of items required for a clear and transparent 
account of what was done and what was found in a research study 


Typically take the form of a checklist, flow diagram or piece of explicit 
text 


Based on available evidence and reflect the consensus opinion of 
experts in a particular field 


Complement advice on scientific writing and journals’ instructions to 
authors 


TRANSPARENT REPORTING of TRIALS 


Some examples include: -A CONSORT 


` STROBE Statement 


apei 
WE il Y 
i Strengthening the reporting of observational studies in epidemiology 





“me PRISMA 


TRANSPARENT REPORTING of SYSTEMATIC REVIEWS and META-ANALYSES 








Example reporting guideline checklists 


CONSORT Statement extension for reporting 


— 4 abstracts of randomized controlled trials 


iW 


This extension to the CONSORT Statement provides a minimum list of essential items, that 
authors should consider when reporting the main results of a randomized trial in any journal 
or conference abstract. 


CONSORT for Abstract Checklist www.consort-statement.org 


Identification of the study as randomized 


PRISMA Statement 2009 — Reporting guideline for systematic 
reviews and meta-analyses 


PRISMA stands for Preferred Reporting Items for Systematic reviews and Meta-Analyses. It is an evidence- 
based minimum set of standards for reporting systematic reviews and meta-analyses. It consists of a 27-item 
checklist and a flow diagram which depicts the flow of information through the different phases of a systematic 
review. 

This guideline replaces the existing QUOROM Statement; journals and other organisations are encouraged to 
update their instructions and resources and refer authors to the new PRISMA guidance. 


PRISMA 2009 Checklist www.prisma-statement.org 


Section / topic # Checklist item 


[Tte |i] Identify the report as a systematic review, meta-analysis, or both. Identify the report as a systematic review, meta-analysis, or both. || 


ABSTRACT 


Structured Provide a structured summary including, as applicable: background; objectives; data 
summary sources; study eligibility criteria, participants, and interventions; study appraisal and 
synthesis methods; results; limitations; conclusions and implications of key findings; 
— Tost =o a = systematic review registration number. 
Participants Eligibility criteria for participants and the settings where the data INTRODUC 
were collected 
CCA Describe the rationale for the review in the context of what is already known. | 
eile Interventions intended for each group e 
Specific objective or hypothesis || 
Clearly defined primary outcome for this report 
z 5 = - = Protocol and 
ae How participants were allocated to interventions address), and, if available, provide registration information including registration 
number. 

Eligibility criteria Specify study characteristics (e.g., PICOS, length of follow-up) and report 
characteristics (e.g., years considered, language, publication status) used as criteria 
for eligibility, giving rationale. 

Information IS all information sources (e.g., databases with dates of coverage, contact with 

sources ME authors to identify additional studies) in the search and date last searched. 


Blinding (masking) Whether or not participants, care givers, and those assessing the 
Present full electronic search strategy for at least one database, including any limits 
used, such that it could be repeated. 


outcomes were blinded to group assignment 
Study selection State the process for selecting studies (i.e., screening, eligibility, included in 
systematic review, and, if applicable, included in the meta-analysis). 


Results 
Numbers randomized Number of participants randomized to each group 
Data collection Describe method of data extraction from reports (e.g., piloted forms, independently, in 
process duplicate) and any processes for obtaining and confirming data from investigators. 
oe | items 


Contact details for the corresponding author 
Description of the trial design (e.g. parallel, cluster, non-inferiority) 


Provide an explicit statement of questions being addressed with reference to 
Participants, interventions, comparisons, outcomes, and study design (PICOS). 


Indicate if a review protocol exists, if and where it can be accessed (e.g., Web 


sai A Number of participants analysed in each group 


For the primary outcome, a result for each group and the estimated 
effect size and ¡ts precision 


List and define all variables for which data were sought (e.g., PICOS, funding 
sources) and any assumptions and simplifications made. 


leal of bias in 12 | Describe methods used for assessing risk of bias of individual studies (including 
individual studies specification of whether this was done at the study or outcome level), and how this 


information is to be used in any data synthesis. 





Cover a range of study types 


Reporting guidelines for main 


Y study types 





Randomised trials CONSORT Extensions Other 
Observational studies STROBE Extensions Other 
Systematic reviews PRISMA Extensions Other 
Case reports CARE Extensions Other 
Qualitative research SROR COREG Other 
Diagnostic / prognostic STARD TRIPOD Other 
studies 
SQUIRE Other 
Economic evaluations CHEERS Other 
Animal pre-clinical studies ARRIVE Other 
Study protocols SPIRIT PRISMA-P Other 
i idell AGREE RIGHT Other 





See all 360 reporting guidelines 





Relevant to different sections of the 
paper/report 







Search for reporting guidelines 


E Browse for reporting guidelines by selecting one of more of these drop.dowmns 


Y 










Study type Clinical area Section of report 
(Pease select eM Please select BEE 


Or search with free text 


Displaying 60 reporting gudelmes found 


Key tepong guideline s shaded green, se hsplayed Srst Show the most recently : 


Gest Practices in Data Analysis and Shanng in Neuroimaging using MRI 














EQUATOR Network 
9 equator Enhancing the QuAlity and 


network Transparency Of health Research 


International initiative to improve the reliability and value of medical research literature 
by promoting transparent and accurate reporting. 


Main focus: 
* Raising awareness 
* Provision of resources 
* Education and training 
* Research 


Established due to growing evidence of serious deficiencies in research literature and 
its effect on the reliability and usability of research results. 


Many reporting guidelines available but awareness and adherence still low. 








EQUATOR website: www.equator-network.org 


© equator Enhancing the QUAIity and Search for reporting guidelines 
network Transparency Of health Research Browse for reporting guidelines by selecting one or more of these drop-downs: 


iCute Library Toolkits Courses &events News Blog Aboutus Contaci te de Clinical area section of report 


Essential resources for writing and publishing health research 














Library for health 2 Reporting guidelines for main r 
research reporting study types tn 
The Library corales a comprehensive searchapie Randomised trials CONSORT Ecersiors One 
Gacansse of reporing guidelines snd a130 links 10 Observational studies STROBE —Exuensions Omer zs Displaying 316 reporting guidelines found. 
oher resources relevant to research reporting Eystemato reviews PRIMA — Exersiors omer sul 
HUE == — == Most recently added records are di first 
kr Qusitebve resesron SROR  COREQ ome ty == 
Diagnostio / prognostio studies STARO © TRIPOD omer E 
Not sure whioh reporting Gusit improvement studies = SQUIRE paci sarea a e : : 3 
(E cestas to uso? SAE ER oe = , Tre Single-CaseR Guideline In BEhavioural Interventions (SCRIBE) 2016 Statement 
Animal preolinios! studies ARRIVE Ores = 
X apalen Study protocols SPRIT PREMIA Omer om sssnsanenenansnsenansuananennnernesevanenensauunsanenenansnsnsananananensnuenesananansuuasnusuananenansnsananananansnenuaenanananenensenenanansnaasnusuanenenanenuauanananansnununuenanananeneaueusuanenenansnusuanananananenunnenenansnenvaueusnananenenene 
Q Viste brary tor See all 316 reporting guidelines Report Consensus on Recordi Endometriosis Surgery: the CORDES statement 
more resouross a] 2 
res 


Developing the Clarity and ness in Reporting: E3-based (CORE) reference user manual for creation of 


3 


ATT 3008/2016 - EQUATOR and PAHO develop practical action pian 
IEC he re aD Ty ana value of ee their scientists in 


medical research imerscre Dy 


Have you remembered everything? 


promoting Tansperent sno secure On 15 March 2016, Or Iveta Simera, Forgetting important details can delay publication and stop your work being cited or replicated. 
ES a a cea Checklists, made by experts and tailored to different study designs, can help. 

Our Tookts suppor diferent user presemacon: Resesrcn papers st 

grous. NL make a diference: how to increase 


This tool will help you find the right checklist for your work from the EQUATOR library. 


impact at he dh University 
imemssionsiizston Seminar heid in 
for anos Washington ... esa More 


2003/2016 - Two great EQUATOR events in Oxford this summer start press ENTER 


A Editors ad EQUATOR wil be at Evidence 








What reporting guidelines are 
available specifically for 
reporting systematic reviews? 





PRISMA Statement 


PRISMA 


TRANSPARENT REPORTING or SYSTEMATIC REVIEWS anp META-ANALYSES 





www.prisma-statement.org 


Preferred Reporting Items for Systematic Reviews and 
Meta-Analyses: The PRISMA Statement 


David Moher [5], Alessandro Liberati, Jennifer Tetzlaff, Douglas G. Altman, The PRISMA Group EJ 
Published: July 21, 2009 e http://dx.doi.org/10.1371/journal. pmed.1000097 


The PRISMA Statement for Reporting Systematic Reviews 
and Meta-Analyses of Studies That Evaluate Health Care 
Interventions: Explanation and Elaboration 


Alessandro Liberati [E], Douglas G. Altman, Jennifer Tetzlaff, Cynthia Mulrow, Peter C. Gatzsche, 
John P A. loannidis, Mike Clarke, P. J. Devereaux, Jos Kleijnen, David Moher 


Published: July 21, 2009 «e http://dx.doi.org/10.1371/journal.pmed.1000100 








PRISMA checklist and flow diagram 


PRISMA Statement 2009 — Reporting guideline for systematic 
reviews and meta-analyses 


PRISMA stands for Preferred Reporting Items for Systematic reviews and Meta-Analyses. It is an evidence- 
based minimum set of standards for reporting systematic reviews and meta-analyses. It consists of a 27-item 
checklist and a flow diagram which depicts the flow of information through the different phases of a systematic 
review. 


This guideline replaces the existing QUOROM Statement; journals and other organisations are encouraged to 
update their instructions and resources and refer authors to the new PRISMA guidance. 


PRISMA 2009 Checklist 





www.prisma-statement.org 


Section / topic # Checklist item 


TITLE 


[Tte [a | Identify the report as a systematic review, meta-analysis, or both. | 
ABSTRACT 


Structured Provide a structured summary including, as applicable: background; objectives; data 

summary sources; study eligibility criteria, participants, and interventions; study appraisal and 
synthesis methods; results; limitations; conclusions and implications of key findings; 
systematic review registration number. 


INTRODUCTION 


| Rationale | 3 | Describe the rationale for the review in the context of what is already known. | | 


Objectives Provide an explicit statement of questions being addressed with reference to 
participants, interventions, comparisons, outcomes, and study design (PICOS). 


METHODS 


Protocol and Indicate if a review protocol exists, if and where it can be accessed (e.g., Web 
registration address), and, if available, provide registration information including registration 
number. 


Eligibility criteria Specify study characteristics (e.g., PICOS, length of follow-up) and report 
characteristics (e.g., years considered, language, publication status) used as criteria 
for eligibility, giving rationale. 


Information 7 | Describe all information sources (e.g., databases with dates of coverage, contact with 

sources study authors to identify additional studies) in the search and date last searched. 
Present full electronic search strategy for at least one database, including any limits 
used, such that it could be repeated. 

Study selection State the process for selecting studies (i.e., screening, eligibility, included in 
systematic review, and, if applicable, included in the meta-analysis). 


Data collection Describe method of data extraction from reports (e.g., piloted forms, independently, in 
process duplicate) and any processes for obtaining and confirming data from investigators. 


| items List and define all variables for which data were sought (e.g., PICOS, funding 
sources) and any assumptions and simplifications made. 


ts of bias in 12 | Describe methods used for assessing risk of bias of individual studies (including 
individual studies specification of whether this was done at the study or outcome level), and how this 


information is to be used in any data synthesis. 


a PRISMA 2009 Flow Diagram 


Records identified through 
tabase — 


Additional records identified 


through other sources 


Records after duplicates removed 


(n=) 


Full-text articles assessed 
for eligibility 
(n=) 


Studies included in 
qualitative synthesis 
(n=) 


Studies included in 
quantitative synthesis 
(meta-analysis) 
(n= ) 


Records excluded 
(n=) 


Full-text articles excluded, 
with reasons 


(n=) 


PRISMA extensions 








PRISMA-P 


PRISMA-P (Preferred Reporting Items for Systematic review and Meta-Analysis Protocols) 2015 checklist: recommended items to 
address in a systematic review protocol* 


Section and topic 


ADMINISTRATIVE INFORMATION 


Title: 
Identification 
Update 

Registration 

Authors: 

Contact 


Contributions 
Amendments 


Support: 
Sources 
Sponsor 
Role of sponsor or funder 


INTRODUCTION 
Rationale 
Objectives 


METHODS 
Eligibility criteria 


Information sources 


Item No 


Preferred reporting items for systematic review and meta-analysis protocols 
(PRISMA-P) 2015 statement 





Identi 
tif David Moher Mí, Larissa Shamseer, Mike Clarke, Davina Ghersi, Alessandro Liberati, Mark Petticrew, Paul Shekelle, Lesley A Stewart and 
E PRISMA-P Group 
If regist 
Systematic Reviews 2015 4:1 DOI: 10.1186/2046-4053-4-1 | © Moher et al.; licensee BioMed Central. 2015 
Provide Partie, nSstiuidodal difiiations =Tlldi. AOG Ui all DIULUCOL duu DTO g DELY al IAIM aGGiess U 


corresponding author 
Describe contributions of protocol authors and identify the guarantor of the review 


If the protocol represents an amendment of a previously completed or published protocol, identify as such and list ch: 
otherwise, state plan for documenting important protocol amendments 


Indicate sources of financial or other support for the review 


Provide name for the review funder and/or sponsor 
Describe roles of funder(s), sponsor(s), and/or institution(s), if any, in developing the protocol 


Describe the rationale for the review in the context of what is already known 


Preferred reporting items for systematic review and meta-analysis 
protocols (PRISMA-P) 2015: elaboration and explanation 


BMJ 2015 ;349 doi: http://dx.doi.org/10.1136/bmj.g7647 (Published 02 January 2015) 
Cite this as: BMJ 2015;349:27647 








Reporting guidelines specifically for reporting 
systematic reviews 


A protocol format for the preparation. registration and publication of systematic reviews of animal intervention 





studies 


A systematic review of systematic reviews and meta-analyses of animal experiments with guidelines for reportin 





Systematic reviews and meta-analysis of preclinical studies: wh form them and how to appraise them 
critically 


Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES 





Finding What Works in Health Care: Standards for Systematic REVICWS. a 
Systematic Reviews 


Bayesian methods in health technology assessment: a review 


RAMESES publication standards: meta-narrative reviews 


The HIGENC HUGE Review Handbook: Version “1:0 SG UIGE MNES TOR Sy ee ee ee 
gene disease association studies 


RAMESES publication standards: realist syntheses 


Cochrane Handbook for S) ¡stematic Reviews Of Interventions Versjon 5.1 Orn nereneenetntnnenerneenernennoneenerneenernennonerneenernennomerneenenennonoeneennnnsnneesnenninarsnannssnnnenonronsnnansanenronesnsenesnennsnronuntaneanenroneunenneaneareneonesneunennenronesneanennennneeneeneaneent 





Meta-analysis of individual participant data: rationale. conduct. and reporting 


A SS 
Systematic review reporting guidelines that include 


recommendations for literature search reporting: 
PRISMA checklist 


INTRODUCTION 

Rationale Describe the rationale for the review in the context of what is already known. 

Objectives Provide an explicit statement of questions being addressed with reference to participants, interventions, comparisons, 
outcomes, and study design (PICOS). 

METHODS 


Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide 
registration information including registration number. 


Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, 
language, publication status) used as criteria for eligibility, giving rationale. 


Protocol and registration 
Eligibility criteria 






be all information sources (e.g., databases with dates of coverage, contact wi 
additional studies) in the search and date last searched. 






repeated. 


State the process for selecting studies (i-e., screening, eligibility, included in systematic review, and, if applicable, 
included in the meta-analysis). 


Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes 
for obtaining and confirming data from investigators. 


Study selection 


= 
© Of on 


Data collection process 


www.prisma-statement.org 





-= 
Systematic review reporting guidelines that include 


recommendations for literature search reporting: 
PRISMA-P checklist 


INTRODUCTION 
‘Rationale 6 ‘Describe the rationale for the review in the context of what is already known 
‘Objectives 7 ‘Provide an explicit statement of the question(s) the review will address with reference to participants, interventions, 
| ¡comparators, and outcomes (PICO) 
METHODS 
Eligibility criteria 8 ¡Specify the study characteristics (such as PICO, study design, setting, time frame) and report characteristics (such as years 
‘considered. language. publication status) to be used as criteria for eligibility for the review 
aetoTmation sources 9 ¡Describe all intended information sources (such as electronic databases, contact with study authors, trial registers or othe: 
grey literature sources) with planned dates of coverage 
10 ‘Present draft of search strategy to be used for at least one electronic database, including planned limits, SUCH 
‘repeated 
¡Study records: 
a: Data management | ila Describe the mechanism(s) that will be used to manage records and data throughout the review... 


www.prisma-statement.org/Extensions/Protocols.aspx 


Systematic review reporting guidelines that include 
recommendations for literature search reporting 


Cochrane Collaboration 


= Handbook Chapter 6.6.2 Provides reporting recommendations for both the review 
and the review protocol: 
http://handbook.cochrane.org/chapter 6/6 6 2 reporting the search process i 
n the review.htm 


= Methodological expectations of Cochrane Intervention Reviews (MECIR): 
Standards for the conduct and reporting of new Cochrane Intervention Reviews. 
R6 provides recommendations for reporting the search in the abstract and R34- 
R39 for reporting the search in the methods section. 


http://www.editorial-unit.cochrane.org/sites/editorial- 





unit.cochrane.org/files/uploads/1MECIR%20standards%20booklet%20V%202.pdf 


Institute of Medicine 

=" Finding What Works in Health Care: Standards for Systematic Reviews. Standards 
for reporting the search outlined in Standard 5.1. 

http://www. nationalacademies.org/hmd/~/media/Files/Report%20Files/2011/Findin 


g-What-Works-in-Health-Care-Standards-for-Systematic- 
Reviews/Standards%20for%20Systematic%20Review%202010%20Insert. pdf 





Examples of reporting guidelines specifically for reporting 
literature searches (but not necessarily for systematic reviews) 
include: 


Atkinson KM, Koenka AC, Sanchez CE, Moshontz H, Cooper H. Reporting standards for 
literature searches and report inclusion criteria: making research syntheses more 
transparent and easy to replicate. Research Synthesis Methods. 2015;6(1): 87—95. 


Niederstadt C, Droste S. Reporting and presenting information retrieval processes: the 
need for optimizing common practice in health technology assessment. Int J Technol 
Assess Health Care. 2010;26(4):450-457. 


Booth A. "Brimful of STARLITE": toward standards for reporting literature searches. J 
Med Libr Assoc. 2006;94(4):421-429, e205. 


Kable AK, Pich J, Maslin-Prothero SE. A structured approach to documenting a search 
strategy for publication: a 12 step guideline for authors. Nurse Educ Today. 
2012;32(8):878-886. 


A 
General reporting items for systematic review literature searches 


include: 


Details of all sources searched including databases, hand 
searches, citation searches, grey literature sources, websites, 
conference proceedings, contact with authors etc. 

Details of all search terms (MeSH and free-text), the fields that 
the terms were searched in, any search filters/hedges used, limits 
applied to the search e.g. date ranges or language or publication 
types, how the terms were combined etc. 

Copy of the entire search strategy for each database (to be 
included in the appendices) 

Date that the search was conducted on each database 

Dates of any update searches conducted for each database 
Details of the search platform used e.g. OVID or EbscoHost 
Details of the exact version of the database used e.g. Embase 
1996 to 2016 Week 36 

The total number of results retrieved individually for each 
database, and combined before and after deduplication 





So why use reporting guidelines 
and what impact can their use 
have? 








Why use reporting guidelines? 


Reporting guidelines are simply an aide memoire - a list of items deemed 
essential for a clear and transparent account of what was done and what was 
found in a research study 


n tay help to: 
improve the accuracy, completeness and reproducibility of research 
reports/papers 
— ensure compliance with journal submission requirements 


— ensure research studies provide a more reliable basis for making 
clinical decisions or for inclusion in further research 


- sae that research results can be transferred into practice more 
quickly 

— improve the quality of the research output of the department / 
institution / organisation 


You can help to improve not only the quali oO and see uent usability of published 
health research but also help advance the global body o health knowledge ultimately 
leading to impro ved patient care! 


Examples of Journal requirements 


thebmj Research ~ Education ~ News&Views~ Campaigns 
















As supplemental files 

= The original protocol for a clinical trial or, if the protocol has been published in an open access online journal, its 
reference and URL. We appreciate that studies sometimes deviate from protocols, but please explain any 
important deviations in the manuscript, particularly those about choice of outcomes and analyses or change in 
sample size. 

The original protocol for an observational study or systematic review, if available. We recommend that protocols 
for randomised trials are written using the SPIRIT checklist. 

For a randomised controlled trial, the appropriate completed CONSORT checklist showing on which page of your 
manuscript each checklist item appears, the CONSORT-style structured abstract, and the CONSORT flowchart 
(CONSORT has several extension statements - for example, for cluster RCTs, pragmatic trials). 

For a randomised controlled trial, a completed TIDieR checklist - this helps to ensure that trial interventions are 
fully described in ways that are reproducible, usable by other clinicians, and clear enough for systematic 
reviewers.and guideline writers. 


Standards of reporting Back to top 


BioMed Central advocates complete and transparent reporting of biomedical and 
biological research. Please refer to the Minimum standards of reporting checklist when 
reporting your research (published in BMC Biology). Exact requirements may vary 
depending on the journal; please refer to the journal's instructions for authors. We 
also strongly recommend that authors refer to the minimum reporting guidelines for 
health research hosted by the EQUATOR Network when preparing their manuscript, 
and the BioSharing Portal for reporting checklists for biological and biomedical 
research, where applicable. Authors should adhere to these guidelines when drafting 
their manuscript, and peer reviewers will be asked to refer to these checklists when 
evaluating such studies. 






Checklists are available for a number of study designs, including: pnd flowchart for a systematic review or meta-analysis of randomised trials and other 


Randomized controlled trials (CONSORT) and protocols (SPIRIT) 

Systematic reviews and meta analyses (EAS) and protocof(ERISMA PID 
Observational studies (STROBE) 

Case reports (CARE) 

Qualitative research (COREQ) 

Diagnostic/prognostic studies (STARD and TRIPOD) 

Economic evaluations (CHEERS) 

Pre-clinical animal studies (ARRIVE) 





we estudies. 
STARD checklist and flowchart for a study of diagnostic accuracy. 
STROBE checklist for an observational study. 

Please use the STROBE extensions, where appropriate: 


= RECORD: The REporting of studies Conducted using Observational Routinely-collected health Data (RECORD) 
Statement 

a RDS: Strengthening the Reporting of Observational Studies in Epidemiology for Respondent-Driven Sampling 
Studies: "STROBE-RDS' Statement 

a ME: STrengthening the Reporting of OBservational studies in Epidemiology — Molecular Epidemiology 
(STROBE-ME) 


= STREGA: STrengthening the REporting of Genetic Association Studies ($ | H E L A N C E i 


= GRIPS for genetic risk prediction studies. 
= CHEERS for an econon e Reports of trials must conform to CONSORT 2010 guidelines, and should be submitted with their prot 


© All reports of randomised trials should include a section entitled Randomisation and masking, withir 


Annals of Internal Medicine «Sisco 
na c e © Cluster-randomised trials must be reported according to CONSORT extended guidelines 


© Randomised trials that report harms must be described according to extended CONSORT guidelines 

© Studies of diagnostic accuracy must be reported according to STARD guidelines 

© Observational studies (cohort, case-control, or cross-sectional designs) must be reported according t 
with their protocols 

© We encourage the registration of all observational studies on a WHO-compliant registry (see Lancet 2 
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Reporting guidelines Follow relevant reporting recommendations. The EQUATOR includes the follo 


UTE AAN AAN RADA © Genetic association studies must be reported according to STREGA guidelj 

elinas 
MOOSE ráporting guidelines for meta-analysis of observational studies a po E : 
ENTREG/eporting guideline for synthesis of qualitative research © Systematic reviews and meta-analyses must be reported according && PRISMA guideline: 


© To find reporting guidelines see http://www.equator-network.org 


ICMJE 


INTERNATIONAL COMMITTEE of ted and sent for peer review simultaneously with the pri- 
| MJE MEDICAL JOURNAL EDITORS mary manuscript. 


2. Reporting Guidelines 
Reporting guidelines have been developed for different 
study designs; examples include CONSORT (www.consort 
-statement.org) for randomized trials, STROBE for obser- 
Confl vational studies (http://strobe-statement.org/), PRISMA 
for systematic reviews and meta-analyses (http://prisma 
-statement.org/), and STARD for studies of diagnostic 
accuracy (www.stard-statement.org/). Journals are encour- 
Publication of Scholarly Work in Medical Journals* aged to ask authors to follow these guidelines because they 
help authors describe the study in enough detail for it to 
be evaluated by editors, reviewers, readers, and other re- 
searchers evaluating the medical literature. Authors of re- 
view manuscripts are encouraged to describe the methods 
used for locating, selecting, extracting, and synthesizing 
data; this is mandatory for systematic reviews. Good 
sources for reporting guidelines are the EQUATOR Net- 
work (www.equator-network.org/home/) and the NLM’s 
www.icmje.org/icmje-recommendations. pdf Research Reporting Guidelines and Initiatives (www.nlm 

-nih.gov/services/research_report_guide.html). 


Recommendations Conflicts of Interest Journals 
Following the ICMJE Recommendations 


Recommendations 


Recommendations for the Conduct, Reporting, Editing, and 





T About the Recommendations A Prepanag a Mamunnee for Loberaon w 


Read the Recommendations for the Conduct, Reporting, Editing, and Use the | 
Publication of Scholarly work in Medical Journals. a disclost 


3. Manuscript Sections 
The following are general requirements for reporting 
within sections of all study designs and manuscript for- 
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Reproducibility and the conduct 


of research 





Data dredging 
Also known as 
p-hacking, this involves 
repeatedly searching 
a dataset or trying 
alternative analyses until 
a ‘significant’ result is 
found. 





Errors 


Technical errors may 
exist within a study, such 
as misidentified reagents 
or computational errors 





Omitting null 
results 
When scientists or 
journals decide not 
to publish studies 
unless results 
are statistically 
significant. 


Issues 





Underspecified 


methods 


A study may be very 
robust, but its methods 
not shared with other 


(E, 





Underpowered 
study 
Statistical power is the 
ability of an analysis 
to detect an effect, if 
the effect exists — an 
underpowered study 
is too small to reliably 
indicate whether or not 
an effect exists. 





Weak 
experimental 
design 
A study may have one 


or more methodological 


flaws that mean it is 
unlikely to produce 
reliable or valid results. 


Possible strategies 


Open data 
Openly sharing results and the underlying data 
with other scientists. 


Pre-registration 
Publicly registering the protocol before a study is 
conducted. 


Collaboration 
Working with other research groups, both 
formally and informally. 


Automation 

Finding technological ways of standardising 
practices, thereby reducing the opportunity for 
human error. 


Open methods 
Publicly publishing the detail of a 
study protocol. 


E] (ys 





le 





scientists in enough 
detail, so others cannot 
precisely replicate it. 


Post-publication review 

Continuing discussion of a study in a public forum Ea 
after it has been published (most are reviewed 

before publication). 


Improving reproducibility will ensure that research is as efficient and productive as 
possible. This figure summarises aspects of the conduct of research that can cause 
irreproducible results, and potential strategies for counteracting poor practice in these 
areas. Overarching factors can further contribute to the causes of irreproducibility, 

but can also drive the implementation of specific measures to address these causes. 
The culture and environment in which research takes place is an important ‘top-down’ 
overarching factor. From a ‘bottom-up’ perspective, continuing education and training 
for researchers can raise awareness and disseminate good practice. 






Reporting guidelines 
Guidelines and checklists that help researchers a 
meet certain criteria when publishing studies. 






Figure taken from the report of the symposium, ‘Reproducibility and reliability of biomedical research’, organised by the Academy 
of Medical Sciences, BBSRC, MRC and Wellcome Trust in April 2015. The full report is available from http:/Avww.acmedsci_ac.uk/ 
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Research impact: learning 
lessons from the REF 


Lesson 5: Researchers who deliver high-quality 
academic research also deliver high-quality 
impact 
















I m pa Ct “The reporting standards of diagnostic accuracy 
studies in the field of non-invasive cardiac imaging 
are satisfactory at best and have improved since 

Ad O ESA ie Eie the introduction of STARD. Those journals that 

advise authors to refer to STARD have 

significantly higher impact factors, and authors 
should be encouraged that journals of relatively 
high impact factors publish diagnostic accuracy 
studies of higher reporting quality.” 
















Reporting standards in cardiac MRI, CT, and SPECT 
diagnostic accuracy studies: analysis of the impact of STARD 
criteria a 








Edd N. Maclean, lan S. Stone, Felix Ceelen, Xabier Garcia-Albeniz, Wieland H. Sommer, Steffen E. Petersen 





DOI: http://dx.doi.ora/10.1093/ehjcijet277 691-700 First published online: 23 January 2014 
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SURGERY 


Impact of the mandatory implementation of reporting 
guidelines on reporting quality in a surgical journal: A before 
and after study 












",,. STROBE compliance following implementation 
of the policy, increased by a statistically significant 
12% (68% to 77%, p=0.00018)...CONSORT 

compliance increased (50% to 70%) as did PRISMA 
compliance (48% to 76%)..." 








Riaz Ahmed Agha, Alexander J Fowle ™ Chris Limb, Katie Whitehurst, Robert Coe, Harkiran Sagoo, Daniyal 
Jafree, Charmilie Chandrakumar, Buket Gundogan 








Calls for development of additional reporting guidelines for reviews: 


A scoping review on the conduct and reporting ot 


Conclusion: “...improvements in reporting and 
scoping reviews conduct are imperative. Further research on 
Andrea C. Tricco 1%, Erin Lillie, Wasifa Zarin, Kelly O'Brien, Heather Colquhoun, Monika Kastner, | SCO ping review methodology is warra nted, 


Danielle Levac, Carmen Ng, Jane Pearson Sharpe, Katherine Wilson, Meghan Kenny, 


Rachel Warren, Charlotte Wilson, Henry T. Stelfox and Sharon E. Straus and in particular, there is need for a guideline 
to standardize reporting." 


BMC Medical Research Methodology BMC series - open, inclusive and trusted 2016 16:15 


DOI: 10.1186/s12874-016-0116-4 © Tricco et al. 2016 
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Is Quality and Completeness of Reporting of 
Systematic Reviews and Meta-Analyses 
Published in High Impact Radiology Journals 
Associated with Citation Rates? 


Christian B. van der Pol’, Matthew D. F. Mcinnes*?*, William Petrcich?, Adam S. Tunis", 
Ramez Hanna‘ 


1 Department of Radiology, University of Ottawa, Ottawa, Ontario, Canada, 2 Clinical Epidemiology 
Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada 


O * mmcinnes @toh.onca 


= Abstract 


Conclusion 


There is a positive correlation between the quality and the completeness of a reported SR 
or MA with citation rate which persists when adjusted for joumal IF and joumal 5-year IF. 


Goren ACCESS 

Citation: van der Pol CB, Mcinnes MDF, Petraich W, 
Tunis AS, Hanna R (2015) ls Quality and 
Completeness of Repating of Systematic Reviews 





